Automatic Formative Assessment Research Articles

Investigating the relationship between math literacy and linguistic synchrony in online mathematical discussions through large‐scale data analytics

AbstractPrevious literature has associated math literacy with linguistic factors such as verbal ability and phonological skills. However, few studies have investigated linguistic synchrony, shown in mathematical discussions. This study modelled math literacy and examined the relationship of math literacy with linguistic synchrony between students and facilitators. We retrieved data from 20,776 online mathematical discussion threads at a secondary school level. First, we assessed students' math literacy based on their discussions and classified them into high‐ and low‐math literacy groups. Then, we conducted Cross‐Recurrence Quantification Analysis (CRQA) to calculate linguistic synchrony within each thread. The result implies that students with high math literacy are more likely to share common words (eg, mathematical terms) with facilitators. At the same time, they would paraphrase the facilitators' words rather than blindly mimic them as the exact sentences or phrases. On the other hand, students with low math literacy tend to use overlapping words with facilitators less frequently and are more likely to repeat the exact same phrases from the facilitators. The findings provide an empirical data analysis and insights into mathematical discussions and linguistic synchrony. In addition, this paper implies the directions to improve online mathematical discussions and foster math literacy. Practitioner notesWhat is already known about this topic Mathematical discussions are known to be an effective way to promote math literacy. Math literacy and linguistic skills have a strong link. Linguistic synchrony is related to better collaboration and common knowledge building. What this paper adds Reveals the relationship between math literacy and linguistic synchrony and deepens the understanding of digital communication in online learning environments. Provides empirical analysis of natural language data in group discussions using CRQA. Conceptualizes linguistic synchrony with three sub‐concepts: linguistic concurrence, predictability, and complexity. Implications for practice and/or policy Educators and practitioners could utilize the automatic formative assessment of math literacy based on the student's language use in mathematical discussions. Educational technology researchers and designers could include CRQA indices and recurrence plots in the dashboard design to provide information to support teachers and learners. Teachers would be able to provide real‐time interventions to promote effective mathematical communication and foster math literacy throughout mathematical discussions.

How can valid and reliable automatic formative assessment predict the acquisition of learning outcomes?

AbstractBackgroundSound learning design should be based on the constructive alignment of intended learning outcomes (LOs), teaching and learning activities and formative and summative assessment. Assessment validity strongly relies on its alignment with LOs. Valid and reliable formative assessment can be analysed as a predictor of students' academic performance, but the question is how significant its predictive power is, and what other elements can affect predictions.ObjectivesOur aim was to investigate the predictive power of formative assessment for summative assessment, measuring the acquisition of LOs.MethodsWe analysed formative assessment results (quizzes, homework), together with log data (video and other material use, class attendance), to determine the most influential predictors and establish a reliable predictive learning analytics model. We used the Random Forest algorithm. The model is based on the data from two university mathematical courses, delivered at different years and levels of study, incorporating 813 students in two consecutive years.Results and ConclusionsOur results show that formative assessment, together with previous summative assessment, is a stronger predictor of summative assessment results than other data on students' engagement. The study pointed to the importance of completeness and quality of data, and clear links between assessment and LOs when making predictions of student results. It suggested that predictions are less reliable for the lowest and the highest performing students. It was noted that other factors can also affect predictions, like the level of LOs, or factors not easily extracted from digital data, like the learning environment and individual students' strategies.

Promoting socioeconomic equity through automatic formative assessment

Ensuring equity in education is a goal for sustainable development. Among the factors that hinder equity, socioeconomic status (SES) has the highest impact on learning Mathematics. This paper addresses the issue of equity at the secondary school level by proposing an approach based on adopting automatic formative assessment (AFA). Carefully designed mathematical activities with interactive feedback were experimented with a sample of 299 students of grade 8 for a school year. A control group of 257 students learned the same topics using traditional methodologies. Part of the sample belonged to low SES. The learning achievement was assessed through pre-and post-tests to understand if the adoption of AFA impacted learning and whether the results depended on the students’ SES. The results show a positive effect of the experimentation (effect size: 0.42). Moreover, the effect size of the experimentation restricted to the low-SES group is high (0.77). In the treatment group, the results do not depend on SES, while in the control group, they do, suggesting that AFA is an equitable approach while traditional instruction risks perpetuating inequalities.

An interactive learning environment to empower engagement in Mathematics

This paper is focused on understanding if the use of interactive activities based on problem solving and automatic formative assessment in a blended learning environment can promote the development of engagement in Mathematics for students of grade 8. In this context, engagement is characterized as a multidimensional construct composed of an emotional component, a cognitive component and a behavioral one. The discussion is based on the results of a didactic experimentation conducted with 299 students of grade 8 who, during one entire school year, had access to an online platform rich in interactive learning materials used in blended modality. Through an initial and a final questionnaire, evolutions in their engagement levels were measured. We found that the initially most disengaged students and those coming from lowest social classes are those who mainly benefitted of the interactive activities in order to improve their engagement levels.

A Tool for Introducing Computer Science with Automatic Formative Assessment

In this paper we present a software platform called Chatbot designed to introduce high school students to Computer Science (CS) concepts in an innovative way: by programming chatbots. A chatbot is a bot that can be programmed to have a conversation with a human or robotic partner in some natural language such as English or Spanish. While programming their chatbots, students use fundamental CS constructs such as variables, conditionals, and finite state automata, among others. Chatbot uses pattern matching, state of the art lemmatization techniques, and finite state automata in order to provide automatic formative assessment to the students. When an error is found, the formative feedback generated is immediate and task-level. We evaluated Chatbot in two observational studies. An online nation-wide competition where more than 10,000 students participated. And, a mandatory in-class 15-lesson pilot course in three high schools. We measured indicators of student engagement (task completion, participation, self reported interest, etc.) and found that girls’ engagement with Chatbot was higher than boys’ for most indicators. Also, in the online competition, the task completion rate for the students that decided to use Chatbot was five times higher than for the students that chose to use the renowned animation and game programming tool Alice. Our results suggest that the availability of automatic formative assessment may have an impact on task completion and other engagement indicators among high school students.

The effect of short formative diagnostic web quizzes with minimal feedback

To help students gauge their understanding of basic concepts and encourage good study habits, we administered short online quizzes that use generic questions in the crucial first few weeks of a course. The purpose of the study was to investigate whether the combination of these web quizzes with generic questions with only binary feedback (right or wrong) would be beneficial for students' learning. We implemented these quizzes in three classes in two different subjects at two different universities, one in Sweden and one in the USA. The students' views on the quizzes' effect on their learning was investigated with surveys and interviews. Almost all students appreciated having these quizzes and 38% of them changed their view on how much they knew of the material covered in the course. Furthermore, over 20% of the students reported altering their study habits as a consequence, in particular studying harder or earlier. In conclusion, this study indicated that short quizzes using generic questions with limited correct/incorrect feedback on each question, have positive effects when administered early in courses. The combination of generic questions and short quizzes could be of value for those contemplating automatic formative assessment, particularly if there is some hesitation with respect to the resources needed for constructing and validating the automatic feedback.

EduMiner: Using text mining for automatic formative assessment

Formative assessment and summative assessment are two widely accepted approaches of assessment. While summative assessment is a typically formal assessment used at the end of a lesson or course, formative assessment is an ongoing process of monitoring learners’ progresses of knowledge construction. Although empirical evidence has acknowledged that formal assessment is indeed superior to summative assessment, current e-learning assessment systems however seldom provide plausible solutions for conducting formative assessment. The major bottleneck of putting formative assessment into practice lies in its labor-intensive and time-consuming nature, which makes it hardly a feasible way of achievement evaluation especially when there are usually a large number of learners in e-learning environment. In this regard, this study developed EduMiner to relieve the burdens imposed on instructors and learners by capitalizing a series of text mining techniques. An empirical study was held to examine effectiveness and to explore outcomes of the features that EduMiner supported. In this study 56 participants enrolling in a “Human Resource Management” course were randomly divided into either experimental groups or control groups. Results of this study indicated that the algorithms introduced in this study serve as a feasible approach for conducting formative assessment in e-learning environment. In addition, learners in experimental groups were highly motivated to phrase the contents with higher-order level of cognition. Therefore a timely feedback of visualized representations is beneficial to facilitate online learners to express more in-depth ideas in discourses.